Effects of Automation Reliability on Human Monitoring Performance

Human decisions to rely on automation of a simulated flight task were monitored in the present study. Participants performed the Multi Attribute Task (MAT) battery previously developed by Comstock and Arneguard (1990). Twenty-eight participants were tested for 7 different reliability levels. Decision accuracy, false alarm rates, and reaction time to simulated automation failures were collected for each participant over nine 10-minute sessions. The results indicated that the rate of automation failures detected varied inversely with automation reliability level. As automation level increased, detection rate decreased thereby indicating automation-induced monitoring inefficiency. However, the mean reaction time of correct detection rate decreased as automation reliability level increased. No significant effect of reliability was obtained for the false alarm rate data. The implications for training and systems design are also outlined.